Papers with LLM-as-a-judge method
Stay Focused: Problem Drift in Multi-Agent Debate (2026.findings-eacl)
Copied to clipboard
| Challenge: | Multi-agent debates have shown promise for solving knowledge and reasoning tasks, but they are limited when solving complex problems that require longer reasoning chains. |
| Approach: | They propose a method to detect problem drift and propose 'driFTJudge' which mitigates 31% of problem drift cases. |
| Outcome: | The proposed method mitigates 31% of problem drift cases and is based on a set of ten tasks across ten different tasks. |
Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Reinforcement learning from human feedback (RLHF) is the primary method for aligning large language models with human preferences. |
| Approach: | They propose to train an Absolute-Rating Multi-Objective Reward Model with multi-dimensional absolute-rating data. |
| Outcome: | The proposed model outperforms the LLM-as-a-judge method on RewardBench . it achieves state-of-the-art performance on the benchmark . |